THE WIDGET EFFECT 

Our National Failure to Acknowledge and 
Act on Differences in Teacher Effectiveness 







The New Teacher Project 



Daniel Weisberg • Susan Sexton • Jennifer Mulhern • David Keeling 



EXECUTIVE SUMMARY 




EXECUTIVE SUMMARY 



Suppose you are a parent determined to make sure your child gets the best possible 
education. You understand intuitively what an ample body of research proves: that your 
child’s education depends to a large extent on the quality of her teachers. Consequently, 
as you begin considering local public schools, you focus on a basic question: who are the best 
teachers, and where do they teach? 

The question is simple enough. There’s just one problem — except for word of mouth from other 
parents, no one can tell you the answers. 

In fact, you would be dismayed to discover that not only can no one tell you which teachers are 
most effective, they also cannot say which are the least effective or which fall in between. Were 
you to examine the district’s teacher evaluation records yourself, you would find that, on paper, 
almost every teacher is a great teacher, even at schools where the chance of a student succeeding 
academically amounts to a coin toss, at best. 

In short, the school district would ask you to trust that it can provide your child a quality 
education, even though it cannot honestly tell you whether it is providing her a quality teacher. 

This is the reality for our public school districts nationwide. Put simply, they fail to distinguish 
great teaching from good, good from fair, and fair from poor. A teacher’s effectiveness — the most 
important factor for schools in improving student achievement — is not measured, recorded, or 
used to inform decision-making in any meaningful way. 
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The Widget Effect 

This report examines our pervasive and longstanding failure to recognize and respond to 
variations in the effectiveness of our teachers. At the heart of the matter are teacher evaluation 
systems, which in theory should serve as the primary mechanism for assessing such variations, 
but in practice tell us little about how one teacher differs from any other, except teachers whose 
performance is so egregiously poor as to warrant dismissal. 

The failure of evaluation systems to provide accurate and credible information about individual 
teachers’ instructional performance sustains and reinforces a phenomenon that we have come to 
call the Widget Effect. The Widget Effect describes the tendency of school districts to assume 
classroom effectiveness is the same from teacher to teacher. This decades-old fallacy fosters an 
environment in which teachers cease to be understood as individual professionals, but rather as 
interchangeable parts. In its denial of individual strengths and weaknesses, it is deeply disrespectful 
to teachers; in its indifference to instructional effectiveness, it gambles with the lives of students. 

Today, the Widget Effect is codified in a policy framework that rarely considers teacher 
effectiveness for key decisions, as illustrated below. 



Where Is Performance a Factor in Important Decisions About Teachers?* 
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The fact that information on teacher performance is almost exclusively used for decisions related 
to teacher remediation and dismissal paints a stark picture: In general, our schools are indifferent 
to instructional effectiveness — except when it comes time to remove a teacher. 



See “Policy Implications of the Widget Effect” for additional information 
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Study Overview 

This report is the product of an extensive research effort spanning 12 districts and 
four states. It reflects survey responses from approximately 15,000 teachers and 
1 ,300 administrators, and it has benefited from the insight of more than 80 local 
and state education officials, teachers union leaders, policymakers and advocates who 
participated in advisory panels in each state, shaping the study design, data collection 
instruments, and findings and recommendations. 

The four states included in the study, Arkansas, Colorado, Illinois and Ohio, employ 
diverse teacher performance management policies. The 1 2 districts studied range in 
size, geographic location, evaluation policies and practices and overall approach to 
teacher performance management. Jonesboro Public Schools, the smallest district 
studied, serves approximately 4,450 students; Chicago Public Schools, the largest, 
serves 413,700. All 12 districts employ some formal evaluation process for teachers, 
but the methods and frequency of evaluation differ. The outcomes, however, are 
strikingly similar. 

Study Sites* 




— AR — 


CO 


1 IL 


| OH 


El Dorado Public Schools 
Jonesboro Public Schools 
Little Rock School District 
Springdale Public Schools 


Denver Public Schools 
Pueblo City Schools 


Chicago Public Schools 
District U-46 (Elgin) 
Rockford Public Schools 


Akron Public Schools 
Cincinnati Public Schools 
Toledo Public Schools 



*For more information on the study sites, please see Methodology. 



Characteristics of the Widget Effect in Teacher Evaluation 

The Widget Effect is characterized by institutional indifference to variations in teacher performance. 
Teacher evaluation systems reflect and reinforce this indifference in several ways. 



All teachers are rated good or great 
In districts that use binary evaluation ratings (generally 
“satisfactory” or “unsatisfactory”), more than 
99 percent of teachers receive the satisfactory rating. 
Districts that use a broader range of rating options do 
little better; in these districts, 94 percent of teachers 
receive one of the top two ratings and less than 
1 percent are rated unsatisfactory. 

Excellence goes unrecognized 
When all teachers are rated good or great, those who 
are truly exceptional cannot be formally identified. 
Fifty-nine percent of teachers and 63 percent of 
administrators say their district is not doing enough 
to identify, compensate, promote and retain the most 
effective teachers. 

Inadequate professional development 
The failure to assess variations in instructional 
effectiveness also precludes districts from identifying 
specific development needs in their teachers. In 
fact, 73 percent of teachers surveyed said their most 
recent evaluation did not identify any development 
areas, and only 45 percent of teachers who did have 
development areas identified said they received useful 
support to improve. 



No special attention to novices 
Inattention to teacher performance and development 
begins from a teacher’s first days in the classroom. 
Though it is widely recognized that teachers are 
least effective in their beginning years, 66 percent 
of novice teachers in districts with multiple ratings 
received a rating greater than “satisfactory” on their 
most recent performance evaluation. Low expectations 
characterize the tenure process as well, with 4 1 percent 
of administrators reporting that they have never “non- 
renewed” a probationary teacher for performance 
concerns in his or her final probationary year. 

Poor performance goes unaddressed 
Despite uniformly positive evaluation ratings, teachers and 
administrators both recognize ineffective teaching in their 
schools. In fact, 81 percent of administrators and 57 percent 
of teachers say there is a tenured teacher in their school 
who is performing poorly, and 43 percent of teachers say 
there is a tenured teacher who should be dismissed for poor 
performance. Troublingly, the percentages are higher in 
high-poverty schools. But district records confirm the 
scarcity of formal dismissals; at least half of the districts 
studied did not dismiss a single non-probationary teacher 
for poor performance in the time period studied (ranging 
from two to five years in each district). 



Flaws in Evaluation Practice and Implementation 

The characteristics above are exacerbated and amplified by cursory evaluation practices and poor implementation. 
Evaluations are short and infrequent (most are based on two or fewer classroom observations, each 60 minutes or less), 
conducted by administrators without extensive training, and influenced by powerful cultural forces — in particular, an 
expectation among teachers that they will be among the vast majority rated as top performers. 

While it is impossible to know whether the system drives the culture or the culture the system, the result is clear — 
evaluation systems fail to differentiate performance among teachers. As a result, teacher effectiveness is largely ignored. 
Excellent teachers cannot be recognized or rewarded, chronically low-performing teachers languish, and the wide 
majority of teachers performing at moderate levels do not get the differentiated support and development they need to 
improve as professionals. 
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Reversing the Widget Effect 

The Widget Effect is deeply ingrained in the fundamental systems and policies that govern 
the teachers in our public schools. Better evaluation systems may offer a partial solution, but 
they will not overcome a culture of indifference to classroom effectiveness. Reversing the 
Widget Effect depends on better information about instructional quality that can be used to 
inform other important decisions that dictate who teaches in our schools. 

0 1 I Adopt a comprehensive performance evaluation system that fairly, 
accurately and credibly differentiates teachers based on their effectiveness 
in promoting student achievement. Teachers should be evaluated based on their 
ability to fulfill their core responsibility as professionals — delivering instruction that 
helps students learn and succeed. This demands clear performance standards, multiple 
rating options, regular monitoring of administrator judgments, and frequent feedback 
to teachers. Furthermore, it requires professional development that is tightly linked to 
performance standards and differentiated based on individual teacher needs. 

The core purpose of evaluation must be maximizing teacher growth and effectiveness, 
not just documenting poor performance as a prelude to dismissal. 

02 I Train administrators and other evaluators in the teacher performance 
evaluation system and hold them accountable for using it effectively. 

The differentiation of teacher effectiveness should be a priority for school 
administrators and one for which they are held accountable. Administrators must 
receive rigorous training and ongoing support so that they can make fair and consistent 
assessments of performance against established standards and provide constructive 
feedback and differentiated support to teachers. 
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03 I Integrate the performance evaluation system with critical human capital policies 
and functions such as teacher assignment, professional development, compensation, 
retention and dismissal. Even the best evaluation system will fail if the information it produces 
is of no consequence. An effective evaluation system must be fully integrated with other district 
systems and policies and a primary factor in decisions such as which teachers receive tenure, how 
teachers are assigned and retained, how teachers are compensated and advanced, what professional 
development teachers receive, and when and how teachers are dismissed. Only by attaching stakes 
to evaluation outcomes will teachers and administrators invest in the hard work of creating a truly 
rigorous and credible evaluation system. 

04 I Adopt dismissal policies that provide lower-stakes options for ineffective 
teachers to exit the district and a system of due process that is fair but efficient. 

If the evaluation system is implemented effectively, unsatisfactory ratings will not be anomalous, 
surprising or without clear justification. Likewise, the identification of development areas and the 
provision of support will be continual. As in other professions, teachers who see significant, credible 
evidence of their own failure to meet standards are likely to exit voluntarily. Districts can facilitate 
this process by providing low-stakes options that enable teachers to leave their positions without 
being exiled. For teachers who must be officially dismissed, an expedited, one-day hearing should be 
sufficient for an arbitrator to determine if the evaluation and development process was followed and 
judgments made in good faith. 

Our recommendations outline a comprehensive approach to improving teacher effectiveness and 
maximizing student learning. If implemented thoroughly and faithfully, we believe they will enable districts 
to understand and manage instructional quality with far greater sophistication. Improved evaluation will 
not only benefit students by driving the systematic improvement and growth of their teachers, but teachers 
themselves, by at last treating them as professionals, not parts. 




<$) About The New Teacher Project 



The New Teacher Project (TNTP) is a national 
nonproTit dedicated to closing the achievement gap by ensuring that poor and minority 
students get outstanding teachers. Founded by teachers in 1997, TNTP partners with 
school districts and states to implement scalable responses to their most acute teacher 
quality challenges. TNTP recruits and trains thousands of exceptional new teachers 
annually, supports school principals in staffing their classrooms, provides teacher 
certification in high-need subjects, and documents the policy barriers that keep students 
from getting the teachers they need. Since its inception, TNTP has trained or hired 
approximately 33,000 teachers, benefiting an estimated 4.8 million students nationwide. 
This report is part of an ongoing series of studies on the policies and practices that 
determine the composition and quality of the nation's teacher workforce. 



For more information, please visit 



www.tntp.org. 
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